Multimodal perceptually rich interfaces in transport

In this paper, I describe research topics in my PhD-project Multimodal perceptually rich interfaces in transport. The project aims at exploring the use of multimodal interfaces, focusing on non-visual sense impressions in complex sociotechnical systems as the operation of high speed crafts. INTRODUCTION My PhD-project Multimodal perceptually rich interfaces in transport is part of an interdisciplinary group looking at the development of user interfaces bridge operations in high-speed crafts. Norway has had one of the world’s biggest fleet of high-speed crafts since the middle of the 1990ies. A typical Norwegian high-speed craft has an operational velocity of about 30 knots and the Norwegian Navy operates a missile torpedo boat with a maximum speed of 60 knots. The challenges for the development of user interfaces for high-speed crafts, are as in other transport areas, numerous. The high speeds require that the operators quickly understand the situation. Complex dynamic environments, such as exposed coastline, nocturnal navigation, insufficiently surveyed and marked shore and locally, occasional high density of traffic makes navigation demanding. The consequences of accidents can be catastrophic. Visual information has, both in product design and in interface design, a vital role. Visual impressions are comprehensive and give us precise information on this dimension of the world. Still, we know that other sense impressions, in other ways, are significant for the way we perceive and understand products and user interfaces. The sound of a car door slamming shut, the simultaneous shake and the smell of the car is an example of important impressions of for example the quality of a product. Physical, touch properties are important to certain forms of interaction (MacLean 2000), and can be expected to have a potential also in the development of user interfaces in high-speed crafts. COMPLEX SOCIO-TECHNOLOGICAL SYSTEMS Systems where people cooperate on a set of tasks, assisted by technology, are often referred to as complex socio-technological systems (Vicente 1999). The operation of crafts the size of high-speed crafts are examples of such systems. The more traditional approach to complex sociotechnological systems has tended to focus on the technical aspects of the systems. The rate of technological development within navigation has increased, and resources spent on technological development have to be followed by an extended training of the navigators using the systems. The convergence of formerly separate systems (navigation with the help of radar, satellite and maps) makes demands on the operators’ understanding of the system. It is therefore necessary to question and challenge the perspectives applied to current developments. MULTIMODAL, PERSEPTUALLY RICH INTERFACES Research on human-technology interaction has traditionally been related to the metaphor of the human cognition as a computer. Succeeding this classical cognitive model several new approaches offers alternative perspectives (Hoff, Øritsland Bjørkli 2002; Dourish 2001). A common feature of these is that they are based on an ecological dynamic relationship between the human and its environment. The sense of touch and kinaesthetic seems to be the sense impression most dependent on such a perspective. Several research findings have shown that the different senses are more tightly coupled than earlier believed, and that they partake in an overlapping interplay (e.g. Ernst & Banks 2002 ; Shimojo & Shams 2001). There is a potential in utilizing this when designing products and interfaces (Monö 1997). To make use of different senses is not at all a new design activity, but rather a natural part of the materialising aspect of design practice. Still, we see that this is not the case in the many electronic products produced, where input and output are reduced to simple buttons, digital screens and the occasional sound signals. The various sense impressions can strengthen and overlap each other, but they can also allow a more complex, nuanced and therefore “rich” dialog with the user. This quality in an interface may be called “perceptually rich” (Hoff 2004). Although many within the field of human factors acknowledge the significance of perceptual richness, few studies have been carried out on the subject. Little is known about suitable theories and methods for the development of rich interfaces. This gives us the following chal-